<html>
<head><meta charset="utf-8"><title>internalize_symbols and incremental artifacts #59535 · t-compiler · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/index.html">t-compiler</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html">internalize_symbols and incremental artifacts #59535</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="180529010"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180529010" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180529010">(Nov 12 2019 at 15:40)</a>:</h4>
<p>hey <span class="user-mention" data-user-id="124287">@mw</span> and/or <span class="user-mention" data-user-id="116015">@Alex Crichton</span> , I wanted to bounce some thoughts off of you two</p>



<a name="180529065"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180529065" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180529065">(Nov 12 2019 at 15:40)</a>:</h4>
<p>in particular, I wanted to talk about a theory I put forth <a href="https://github.com/rust-lang/rust/issues/59535#issuecomment-552894688" target="_blank" title="https://github.com/rust-lang/rust/issues/59535#issuecomment-552894688">here</a>.</p>



<a name="180529209"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180529209" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180529209">(Nov 12 2019 at 15:42)</a>:</h4>
<p>We currently make decisions about whether to change a symbol's Linkage from External to Internal based on the set of accessors we currently observe. However, I theorize that old incremental codegen artifacts may be relying on a symbol's previous classification as External</p>



<a name="180529300"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180529300" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180529300">(Nov 12 2019 at 15:43)</a>:</h4>
<p>so I was wondering: can/should we have a rule that if a symbol was considered External in any previous compile, then an incremental compilation is forced to keep it External?</p>



<a name="180531826"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180531826" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180531826">(Nov 12 2019 at 16:05)</a>:</h4>
<p>(this is not the only possible solution to the theorized problem. For example, instead of constraining which symbols can be marked Internal based on previous compiles, we could instead force the codegen artifacts to be recompiled rather than reused; but doing this well would require determiniing which codegen artifacts depend on the symbols previously marked as External, and I do not know how hard that is to implement.)</p>



<a name="180536052"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180536052" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180536052">(Nov 12 2019 at 16:46)</a>:</h4>
<p>mw will probably have more to say about this than I, but at least from my perspective I would say that artifacts from a previous compile should only ever accelerate future compiles, never actually change the future compile</p>



<a name="180536070"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180536070" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180536070">(Nov 12 2019 at 16:46)</a>:</h4>
<p>so that way no matter what your cache looks like <code>cargo build</code> always produces the same thing</p>



<a name="180536087"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180536087" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180536087">(Nov 12 2019 at 16:46)</a>:</h4>
<p>so I'd probably say that this should lean on the side of recompiling more rather than keeping something External</p>



<a name="180536108"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180536108" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Alex Crichton <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180536108">(Nov 12 2019 at 16:46)</a>:</h4>
<p>although if that's too onerous, then we could also perhaps start exporting more symbols in incremental mode</p>



<a name="180582262"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180582262" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Zoxc <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180582262">(Nov 13 2019 at 01:55)</a>:</h4>
<blockquote>
<p>However, I theorize that old incremental codegen artifacts may be relying on a symbol's previous classification as External</p>
</blockquote>
<p>Old artifacts must not be used if they are not the same as the artifacts produced by the current compilation session, so what the previous classification was should not matter. In theory at least, there might be bugs in the compiler.</p>



<a name="180589633"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180589633" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180589633">(Nov 13 2019 at 04:12)</a>:</h4>
<p>my current inclination is to try turning off the internalize_symbols optimization if we have incremental compilation turned on...</p>



<a name="180589696"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180589696" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180589696">(Nov 13 2019 at 04:14)</a>:</h4>
<p>(Though I worry that may cause too great a regression in code quality, especially code size...)</p>



<a name="180589819"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180589819" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180589819">(Nov 13 2019 at 04:17)</a>:</h4>
<p>One question related to this: Am I right that LLVM <em>could</em> be inlining a function that comes from a <em>different</em> codegen unit in the current crate? To be honest that part of the story does not match my mental model of how LLVM optimization works.</p>



<a name="180589998"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180589998" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180589998">(Nov 13 2019 at 04:21)</a>:</h4>
<p>Hmm: is there some #[rustc_attribute] I can use to force items to be assigned to particular codegen units? That would help me write a test case that would allow exploring the behavior here without relying on the oddities of this particular test.</p>



<a name="180613483"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180613483" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180613483">(Nov 13 2019 at 10:57)</a>:</h4>
<p>/me is busy at the moment. Will take a closer look on Friday.</p>



<a name="180615390"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180615390" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180615390">(Nov 13 2019 at 11:22)</a>:</h4>
<blockquote>
<p>One question related to this: Am I right that LLVM <em>could</em> be inlining a function that comes from a <em>different</em> codegen unit in the current crate? To be honest that part of the story does not match my mental model of how LLVM optimization works.</p>
</blockquote>
<p>further update on this detail: I think the inlining here is occurring during LTO</p>



<a name="180617486"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180617486" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180617486">(Nov 13 2019 at 11:54)</a>:</h4>
<blockquote>
<p>Hmm: is there some #[rustc_attribute] I can use to force items to be assigned to particular codegen units? That would help me write a test case that would allow exploring the behavior here without relying on the oddities of this particular test.</p>
</blockquote>
<p>(maybe getting this effect is supposed to be as simple as choosing the right set of <code>mod</code> items. still looking; but I think an explicit attribute could potentially be a good thing.)</p>



<a name="180713730"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180713730" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180713730">(Nov 14 2019 at 09:35)</a>:</h4>
<p>I left more notes in the bug (<a href="https://github.com/rust-lang/rust/issues/59535" target="_blank" title="https://github.com/rust-lang/rust/issues/59535">#59535</a>). I now better understand what's happening. Part of the problem is that we are reusing the post-ThinLTO object file for a codegen-unit (and that's what holds the result of inlining <code>fn B</code> from a different codegen-unit); so <code>fn B</code>'s call to <code>fn A</code> has been inlined into that object file. But independently, due to changes elsewhere in the source code, we reclassified <code>fn A</code> as internal, and subsequently optimized away the definition of <code>fn A</code>.</p>



<a name="180713920"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180713920" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180713920">(Nov 14 2019 at 09:38)</a>:</h4>
<p>I am looking over <a href="https://github.com/rust-lang/rust/blob/5e380b797b22e5361a43b2b82f6278df17d89f3e/src/librustc_codegen_llvm/back/lto.rs#L499" target="_blank" title="https://github.com/rust-lang/rust/blob/5e380b797b22e5361a43b2b82f6278df17d89f3e/src/librustc_codegen_llvm/back/lto.rs#L499">lto.rs</a> now, and also trying to understand the LLVM LTO docs. One oddity is that the erroneously reused codegen unit claims it does not import from any modules; but surely the call to the other function should be considered an import?</p>



<a name="180846860"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180846860" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180846860">(Nov 15 2019 at 16:13)</a>:</h4>
<p>Argh, sorry I didn't get to this before the weekend ...</p>



<a name="180847130"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847130" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847130">(Nov 15 2019 at 16:16)</a>:</h4>
<p>I think there is a small bug somewhere here, as opposed to a conceptual problems because otherwise we'd see much more of this</p>



<a name="180847183"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847183" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847183">(Nov 15 2019 at 16:17)</a>:</h4>
<p>and I remember that I ran into problems immediately when not handling symbol internalization properly while initially implementing incremental ThinLTO</p>



<a name="180847290"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847290" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847290">(Nov 15 2019 at 16:18)</a>:</h4>
<p>yes I am working my way gradually towards the camp of "small bug somewhere" rather than "conceptual problem"</p>



<a name="180847299"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847299" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847299">(Nov 15 2019 at 16:18)</a>:</h4>
<p>it just took me a while to put all the pieces together</p>



<a name="180847322"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847322" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847322">(Nov 15 2019 at 16:19)</a>:</h4>
<p>at this point I'm not sure whether the small bug is on our end or within LLVM.</p>



<a name="180847342"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847342" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847342">(Nov 15 2019 at 16:19)</a>:</h4>
<p>yeah, it's non trivial and should be better documented</p>



<a name="180847363"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847363" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847363">(Nov 15 2019 at 16:19)</a>:</h4>
<p>I was pretty sure for a while it was our fault, but then when I was trying to walk through some of the execution, the behavior  I saw from LLVM was pretty strange</p>



<a name="180847369"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847369" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847369">(Nov 15 2019 at 16:19)</a>:</h4>
<p>I'll reserved a 2h time slot for this on Tuesday</p>



<a name="180847374"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847374" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847374">(Nov 15 2019 at 16:19)</a>:</h4>
<p>anyway don't worry about delaying this</p>



<a name="180847445"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847445" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847445">(Nov 15 2019 at 16:20)</a>:</h4>
<p><span aria-label="+1" class="emoji emoji-1f44d" role="img" title="+1">:+1:</span></p>



<a name="180847462"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847462" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847462">(Nov 15 2019 at 16:20)</a>:</h4>
<p>I spent a long time on it because I know how painful incremetnal bugs are, but also, we have so many that we <em>don't</em> have reproducible test cases for</p>



<a name="180847481"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847481" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847481">(Nov 15 2019 at 16:20)</a>:</h4>
<p>so the fact that we could actually reproduce this one got me very excited.</p>



<a name="180847497"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847497" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847497">(Nov 15 2019 at 16:20)</a>:</h4>
<p>:D</p>



<a name="180847501"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/180847501" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#180847501">(Nov 15 2019 at 16:20)</a>:</h4>
<p>perhaps over enthusiastically so</p>



<a name="181105791"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181105791" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181105791">(Nov 19 2019 at 12:20)</a>:</h4>
<p>this is incredibly weird :)</p>



<a name="181105815"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181105815" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181105815">(Nov 19 2019 at 12:20)</a>:</h4>
<p>I've been able to reproduce with <code>doit2.sh</code> from <a href="https://github.com/pnkfelix/rust-issue-59535" target="_blank" title="https://github.com/pnkfelix/rust-issue-59535">https://github.com/pnkfelix/rust-issue-59535</a></p>



<a name="181105854"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181105854" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181105854">(Nov 19 2019 at 12:21)</a>:</h4>
<p>but I have no clue what's going on yet</p>



<a name="181105862"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181105862" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181105862">(Nov 19 2019 at 12:21)</a>:</h4>
<p>it looks like this has nothing to do with ThinLTO</p>



<a name="181106332"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181106332" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181106332">(Nov 19 2019 at 12:29)</a>:</h4>
<p>the <code>volatile</code> cgu for <code>rubble::ble</code> declares <code>&lt;rubble[317d481089b8c8fe]::ble::Duration as core[5e7ea8cd6850b425]::fmt::Display&gt;::fmt</code> in the initial version</p>



<a name="181106409"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181106409" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181106409">(Nov 19 2019 at 12:30)</a>:</h4>
<p>in the next version it should declare <code>&lt;rubble[317d481089b8c8fe]::ble::Duration as core[5e7ea8cd6850b425]::fmt::Debug&gt;::fmt</code> but the CGU is exactly the same version as initially</p>



<a name="181106512"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181106512" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181106512">(Nov 19 2019 at 12:32)</a>:</h4>
<p>the internalizer <em>correctly</em> makes the symbol for <code>&lt;rubble[317d481089b8c8fe]::ble::Duration as core[5e7ea8cd6850b425]::fmt::Display&gt;::fmt</code> internal, but it looks like the <code>volatile</code> object file for <code>rubble::ble</code> is not recompiled, although the MIR it is generated from should have changed.</p>



<a name="181106576"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181106576" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181106576">(Nov 19 2019 at 12:33)</a>:</h4>
<p>I'm trying to debug further but for that I need a local build of rustc that can handle the <code>thumbv7em-none-eabi</code> target</p>



<a name="181106624"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181106624" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181106624">(Nov 19 2019 at 12:34)</a>:</h4>
<p>how hard can that be :P</p>



<a name="181107157"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107157" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107157">(Nov 19 2019 at 12:42)</a>:</h4>
<p>Oh I should have warned you about that target dependence</p>



<a name="181107161"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107161" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107161">(Nov 19 2019 at 12:42)</a>:</h4>
<p>what did I end up having to do there ...</p>



<a name="181107234"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107234" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107234">(Nov 19 2019 at 12:43)</a>:</h4>
<p>I'm curious you say the MIR should have changed. I could have sworn that I validated that the inlining in question was happening during a LTO optimization pass</p>



<a name="181107297"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107297" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107297">(Nov 19 2019 at 12:44)</a>:</h4>
<p>regarding the target dependence, I know that I installed thumbv7em-none-eabi via <code>rustup</code>, but that's not the local build</p>



<a name="181107316"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107316" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107316">(Nov 19 2019 at 12:45)</a>:</h4>
<p>if I compile v2 with an empty cache, the volatile cgu does not reference the function in question anymore</p>



<a name="181107326"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107326" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107326">(Nov 19 2019 at 12:45)</a>:</h4>
<p>it references a different function instead</p>



<a name="181107332"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107332" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107332">(Nov 19 2019 at 12:45)</a>:</h4>
<p>and everything works</p>



<a name="181107417"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107417" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107417">(Nov 19 2019 at 12:46)</a>:</h4>
<p>but let me take another look...</p>



<a name="181107437"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107437" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107437">(Nov 19 2019 at 12:47)</a>:</h4>
<p>okay, this is how I built: <code>time python /home/pnkfelix/Dev/Mozilla/rust.git/x.py dist --stage 2   --target thumbv7em-none-eabi</code></p>



<a name="181107464"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107464" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107464">(Nov 19 2019 at 12:47)</a>:</h4>
<p>though that might need to be preceded by a <code>python x.py build --stage 2 --target thumbv7em-none-eabi</code></p>



<a name="181107485"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107485" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107485">(Nov 19 2019 at 12:48)</a>:</h4>
<p>(for some reason I vaguely recall being unhappy about my attempts to go directly to <code>dist</code>)</p>



<a name="181107590"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107590" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107590">(Nov 19 2019 at 12:49)</a>:</h4>
<p>I'm getting <code>thread 'main' panicked at 'All the *-none-* and nvptx* targets are no-std targets', src/bootstrap/sanity.rs:186:17</code> with that</p>



<a name="181107665"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107665" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107665">(Nov 19 2019 at 12:50)</a>:</h4>
<p>regarding the obj file, already the "no-opt" version of the LLVM IR should look different, so this is before ThinLTO, unless I'm overlooking something</p>



<a name="181107671"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107671" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107671">(Nov 19 2019 at 12:50)</a>:</h4>
<p>hmm.</p>



<a name="181107770"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107770" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107770">(Nov 19 2019 at 12:52)</a>:</h4>
<p>when you say "the volatile cgu", is that referring to <code>mks4f5lxe2i4sqc</code> ? (I am assuming the names are stable between your platform and my own, but perhaps  that is an invalid assumption)</p>



<a name="181107829"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107829" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107829">(Nov 19 2019 at 12:53)</a>:</h4>
<p>I'm compiling with <code>-Zhuman-readable-cgu-names</code>, which gives me names corresponding to the module path</p>



<a name="181107847"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107847" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107847">(Nov 19 2019 at 12:53)</a>:</h4>
<p>ah hmm</p>



<a name="181107849"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107849" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107849">(Nov 19 2019 at 12:53)</a>:</h4>
<p>e.g. rubble.rubble.7rcbfp3g-ble.volatile.rcgu.bc</p>



<a name="181107850"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107850" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107850">(Nov 19 2019 at 12:53)</a>:</h4>
<p>didn't know about that flag</p>



<a name="181107872"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107872" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107872">(Nov 19 2019 at 12:54)</a>:</h4>
<p>the above can be decoded as <code>rubble[7rcbfp3g]::ble</code></p>



<a name="181107941"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107941" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107941">(Nov 19 2019 at 12:54)</a>:</h4>
<p>where the version with <code>volatile</code> will contain the code corresponding to generic instances</p>



<a name="181107958"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181107958" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181107958">(Nov 19 2019 at 12:55)</a>:</h4>
<p>and the one without <code>volatile</code> will contain the code for non-generic stuff from that module</p>



<a name="181108607"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181108607" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181108607">(Nov 19 2019 at 13:04)</a>:</h4>
<p>I don't understand the no_std logic in x.py</p>



<a name="181109122"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109122" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109122">(Nov 19 2019 at 13:10)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> or <span class="user-mention" data-user-id="121055">@Pietro Albini</span>, do you happen to know how I can locally build a rustc that can target <code>thumbv7em-none-eabi</code>?</p>



<a name="181109138"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109138" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109138">(Nov 19 2019 at 13:10)</a>:</h4>
<p>should be able to <code>x.py build src/libstd --target thumbv7em-none-eabi</code></p>



<a name="181109149"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109149" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109149">(Nov 19 2019 at 13:10)</a>:</h4>
<p>unless you need C compiler or something</p>



<a name="181109151"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109151" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109151">(Nov 19 2019 at 13:10)</a>:</h4>
<p><code>x.py dist --stage 2   --target thumbv7em-none-eabi</code> gives me <code>thread 'main' panicked at 'All the *-none-* and nvptx* targets are no-std targets', src/bootstrap/sanity.rs:186:17</code></p>



<a name="181109159"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109159" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109159">(Nov 19 2019 at 13:11)</a>:</h4>
<p>maybe if I delete my config.toml?</p>



<a name="181109181"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109181" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109181">(Nov 19 2019 at 13:11)</a>:</h4>
<blockquote>
<p>should be able to <code>x.py build src/libstd --target thumbv7em-none-eabi</code></p>
</blockquote>
<p>I know this was not sufficient for me</p>



<a name="181109189"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109189" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109189">(Nov 19 2019 at 13:11)</a>:</h4>
<p>you can also wait for a while and just use dist-various-1</p>



<a name="181109193"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109193" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109193">(Nov 19 2019 at 13:11)</a>:</h4>
<p>that'll definitely work</p>



<a name="181109196"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109196" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109196">(Nov 19 2019 at 13:11)</a>:</h4>
<p>I definitely had to do <em>something</em> beyond <code>x.py build</code> to get a libstd built (and put into a working location) for the target</p>



<a name="181109201"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109201" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109201">(Nov 19 2019 at 13:11)</a>:</h4>
<p><code>./src/ci/docker/run.sh dist-various-1</code></p>



<a name="181109298"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109298" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109298">(Nov 19 2019 at 13:13)</a>:</h4>
<p>Re-running <code>x.py build src/libstd --target thumbv7em-none-eabi</code> without a <code>config.toml</code> now ...</p>



<a name="181109327"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109327" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109327">(Nov 19 2019 at 13:13)</a>:</h4>
<p>(I think) I have the correct GCC arm toolchain installed already</p>



<a name="181109332"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109332" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109332">(Nov 19 2019 at 13:13)</a>:</h4>
<p>we don't seem to do anything special in that docker container so I <em>would</em> expect that to work</p>



<a name="181109392"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109392" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109392">(Nov 19 2019 at 13:14)</a>:</h4>
<p><a href="https://github.com/rust-lang/rust/blob/a0d40f8bdfcc3c28355467973f97fd4c45ac5876/src/ci/docker/dist-various-1/Dockerfile#L170" target="_blank" title="https://github.com/rust-lang/rust/blob/a0d40f8bdfcc3c28355467973f97fd4c45ac5876/src/ci/docker/dist-various-1/Dockerfile#L170">https://github.com/rust-lang/rust/blob/a0d40f8bdfcc3c28355467973f97fd4c45ac5876/src/ci/docker/dist-various-1/Dockerfile#L170</a></p>



<a name="181109543"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109543" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109543">(Nov 19 2019 at 13:16)</a>:</h4>
<p>I'll try a clean build with <code>x.py build src/libstd --target thumbv7em-none-eabi</code> to try to get a better record of what went wrong there. I had assume it was user error back when I found that to be insufficient.</p>



<a name="181109702"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109702" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109702">(Nov 19 2019 at 13:18)</a>:</h4>
<p>/me is still waiting for build ...</p>



<a name="181109727"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109727" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109727">(Nov 19 2019 at 13:18)</a>:</h4>
<p>(I'm glad I have a rather snappy desktop machine available)</p>



<a name="181109847"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109847" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109847">(Nov 19 2019 at 13:20)</a>:</h4>
<p>OK, it failed with something different</p>



<a name="181109864"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109864" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109864">(Nov 19 2019 at 13:20)</a>:</h4>
<p>7 mins for a stage 2 build is not bad though!</p>



<a name="181109894"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181109894" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181109894">(Nov 19 2019 at 13:21)</a>:</h4>
<div class="codehilite"><pre><span></span>cargo:warning=/xoxo/3-rust/src/llvm-project/compiler-rt/lib/builtins/int_util.c:57:10: fatal error: stdlib.h: No such file or directory
cargo:warning=   57 | #include &lt;stdlib.h&gt;
cargo:warning=      |          ^~~~~~~~~~
cargo:warning=compilation terminated.
</pre></div>



<a name="181110243"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181110243" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181110243">(Nov 19 2019 at 13:25)</a>:</h4>
<p>hm that is ... unexpected</p>



<a name="181110268"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181110268" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181110268">(Nov 19 2019 at 13:25)</a>:</h4>
<p>dist-various-1 would be my recommendation (you can comment a bunch in the relevant dockerfile to get it faster)</p>



<a name="181110589"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181110589" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181110589">(Nov 19 2019 at 13:29)</a>:</h4>
<p>I had to install <code>arm-none-eabi-newlib</code> on my system, now it works</p>



<a name="181110659"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181110659" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181110659">(Nov 19 2019 at 13:30)</a>:</h4>
<p>why would the GCC package not include that? weird</p>



<a name="181110680"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181110680" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181110680">(Nov 19 2019 at 13:30)</a>:</h4>
<p>now I need to get <code>rust-lld</code> working :(</p>



<a name="181110800"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181110800" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181110800">(Nov 19 2019 at 13:32)</a>:</h4>
<p>Is that not just a <code>rustup</code> thing?</p>



<a name="181110807"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181110807" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181110807">(Nov 19 2019 at 13:32)</a>:</h4>
<p>you are really making wish I had kept a full transcript of my full debugging session...</p>



<a name="181111528"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181111528" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181111528">(Nov 19 2019 at 13:41)</a>:</h4>
<p>using rust-lld for the rustup's nightly now fails compiling the support libraries... frustrating</p>



<a name="181112217"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181112217" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181112217">(Nov 19 2019 at 13:51)</a>:</h4>
<p>now my reproduction script seems broken even with nightly :(</p>



<a name="181112233"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181112233" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181112233">(Nov 19 2019 at 13:51)</a>:</h4>
<p>(btw, thanks for you help, <span class="user-mention" data-user-id="116122">@simulacrum</span>!)</p>



<a name="181113471"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181113471" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181113471">(Nov 19 2019 at 14:07)</a>:</h4>
<p>Hm, so the thing that changes is <code>&lt;&amp;rubble[317d481089b8c8fe]::ble::Duration as core[5e7ea8cd6850b425]::fmt::Debug&gt;::fmt</code>. Is this maybe something completely compiler generated for which we never instantiate any MIR?</p>



<a name="181113472"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181113472" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181113472">(Nov 19 2019 at 14:07)</a>:</h4>
<div class="codehilite"><pre><span></span><span class="c">; Function Attrs: nounwind optsize</span>
<span class="k">define</span> <span class="k">hidden</span> <span class="k">zeroext</span> <span class="k">i1</span> <span class="vg">@_RNvXsL_NtCs86ZAEM1TFWr_4core3fmtRNtNtCs4fqI2P2rA04_6rubble3ble8DurationNtB5_5Debug3fmtBz_</span><span class="p">(</span><span class="k">i32</span><span class="p">**</span> <span class="k">noalias</span> <span class="k">nocapture</span> <span class="k">readonly</span> <span class="k">align</span> <span class="m">4</span> <span class="k">dereferenceable</span><span class="p">(</span><span class="m">4</span><span class="p">),</span> <span class="nv">%&quot;core::fmt::Formatter&quot;</span><span class="p">*</span> <span class="k">align</span> <span class="m">4</span> <span class="k">dereferenceable</span><span class="p">(</span><span class="m">52</span><span class="p">))</span> <span class="k">unnamed_addr</span> <span class="vg">#0</span> <span class="p">{</span>
  <span class="nv nv-Anonymous">%3</span> <span class="p">=</span> <span class="k">load</span> <span class="k">i32</span><span class="p">*,</span> <span class="k">i32</span><span class="p">**</span> <span class="nv nv-Anonymous">%0</span><span class="p">,</span> <span class="k">align</span> <span class="m">4</span><span class="p">,</span> <span class="nv">!nonnull</span> <span class="nv nv-Anonymous">!0</span>
  <span class="nv nv-Anonymous">%4</span> <span class="p">=</span> <span class="k">tail</span> <span class="k">call</span> <span class="k">zeroext</span> <span class="k">i1</span> <span class="vg">@_RNvXs_NtCs4fqI2P2rA04_6rubble3bleNtB4_8DurationNtNtCs86ZAEM1TFWr_4core3fmt5Debug3fmt</span><span class="p">(</span><span class="k">i32</span><span class="p">*</span> <span class="k">noalias</span> <span class="k">nonnull</span> <span class="k">readonly</span> <span class="k">align</span> <span class="m">4</span> <span class="k">dereferenceable</span><span class="p">(</span><span class="m">4</span><span class="p">)</span> <span class="nv nv-Anonymous">%3</span><span class="p">,</span> <span class="nv">%&quot;core::fmt::Formatter&quot;</span><span class="p">*</span> <span class="k">nonnull</span> <span class="k">align</span> <span class="m">4</span> <span class="k">dereferenceable</span><span class="p">(</span><span class="m">52</span><span class="p">)</span> <span class="nv nv-Anonymous">%1</span><span class="p">)</span>
  <span class="k">ret</span> <span class="k">i1</span> <span class="nv nv-Anonymous">%4</span>
<span class="p">}</span>
</pre></div>



<a name="181113531"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181113531" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181113531">(Nov 19 2019 at 14:08)</a>:</h4>
<p>Might be a shim that incr. comp. is somehow missing?</p>



<a name="181113551"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181113551" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181113551">(Nov 19 2019 at 14:08)</a>:</h4>
<p>/me needs to run now unfortunately</p>



<a name="181291875"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181291875" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181291875">(Nov 21 2019 at 10:28)</a>:</h4>
<p><span class="user-mention" data-user-id="116083">@pnkfelix</span> I suggest trying to find out where the above LLVM IR is generated and what MIR it is generated from if any. You can use <code>-Zprint-mono-items=lazy</code> to get additional information.</p>



<a name="181291892"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181291892" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181291892">(Nov 21 2019 at 10:28)</a>:</h4>
<p>okay thanks</p>



<a name="181291924"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181291924" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181291924">(Nov 21 2019 at 10:29)</a>:</h4>
<p>I did have a question for you</p>



<a name="181291929"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181291929" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181291929">(Nov 21 2019 at 10:29)</a>:</h4>
<p>related to this</p>



<a name="181292038"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181292038" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181292038">(Nov 21 2019 at 10:30)</a>:</h4>
<p>but I don't think I'm going to be able to re-establish the context for my Q quickly enough...</p>



<a name="181292072"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181292072" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181292072">(Nov 21 2019 at 10:31)</a>:</h4>
<p>the big picture of the Question was that I had tracked down a spot in the rust &lt;-&gt; LLVM interactions</p>



<a name="181292085"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181292085" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181292085">(Nov 21 2019 at 10:31)</a>:</h4>
<p>where Rust was querying LLVM for a list of the modules that a given cgu depended upon</p>



<a name="181292118"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181292118" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181292118">(Nov 21 2019 at 10:31)</a>:</h4>
<p>and I remember being surprised to see an empty list for the case in question, something where the given cgu was making a call out to a function in another cgu</p>



<a name="181292170"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181292170" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181292170">(Nov 21 2019 at 10:32)</a>:</h4>
<p>So that was what I had wanted to ask you about, the expected semantics of what that list contained</p>



<a name="181292188"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181292188" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181292188">(Nov 21 2019 at 10:32)</a>:</h4>
<p>but now I cannot find any of the details quickly. (I restarted the relevant emacs sessions so that I could reboot the machine, and this effectively obliterates my brain's working memory.)</p>



<a name="181292616"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181292616" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181292616">(Nov 21 2019 at 10:39)</a>:</h4>
<p>ah, wait, thank goodness, my working directory still has my instrumentation in it</p>



<a name="181292736"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181292736" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181292736">(Nov 21 2019 at 10:41)</a>:</h4>
<p><code>rustc_codegen_llvm::back::lto</code>  is the relevant module in rustc</p>



<a name="181292804"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181292804" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181292804">(Nov 21 2019 at 10:42)</a>:</h4>
<p>and I added some code to instrument this loop: <a href="https://github.com/rust-lang/rust/blob/f1b882b55805c342e46ee4ca3beeef1d1fa2044b/src/librustc_codegen_llvm/back/lto.rs#L509" target="_blank" title="https://github.com/rust-lang/rust/blob/f1b882b55805c342e46ee4ca3beeef1d1fa2044b/src/librustc_codegen_llvm/back/lto.rs#L509">https://github.com/rust-lang/rust/blob/f1b882b55805c342e46ee4ca3beeef1d1fa2044b/src/librustc_codegen_llvm/back/lto.rs#L509</a></p>



<a name="181292900"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181292900" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181292900">(Nov 21 2019 at 10:43)</a>:</h4>
<p>but of course I have since lost my build of the <code>libcore</code> for the thumbv7em target. <span aria-label="sad" class="emoji emoji-2639" role="img" title="sad">:sad:</span></p>



<a name="181293037"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181293037" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181293037">(Nov 21 2019 at 10:45)</a>:</h4>
<p>well, I guess my question is still relevant, nonetheless: should we expect <code>import_map.modules_imported_by(module_name)</code> to include modules that hold the definitions for functions being called <em>from</em> <code>module_name</code> ? Or do I misunderstand what "imported_by" is meant to mean here?</p>



<a name="181293111"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181293111" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181293111">(Nov 21 2019 at 10:46)</a>:</h4>
<p>(because it is just those sorts of functions, undergoing changes, that I would expect would lead us to conclude that we cannot reuse the post-ThinLTO-optimized artifact.)</p>



<a name="181298766"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181298766" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181298766">(Nov 21 2019 at 12:20)</a>:</h4>
<p>iirc, <code>import_map.modules_imported_by(module_name)</code> should contain all LLVM modules that ThinLTO actually pulled definitions in from. So it is ThinLTO specific. There is another "import step" that is done already in rustc: All functions that are marked as <code>#[inline]</code> are instantiated in all  LLVM module that contain calls to them.</p>



<a name="181298909"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181298909" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181298909">(Nov 21 2019 at 12:22)</a>:</h4>
<p>so there can be functions that get "inlined across modules" before ThinLTO is invoked.</p>



<a name="181298953"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181298953" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181298953">(Nov 21 2019 at 12:23)</a>:</h4>
<p>but in that case we actually don't inline across module boundaries, we just instantiate internal copies of a given function in each calling module.</p>



<a name="181299049"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181299049" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181299049">(Nov 21 2019 at 12:24)</a>:</h4>
<p>this functionality comes from a time before there was ThinLTO and it yields better results than ThinLTO, so we had to keep it.</p>



<a name="181299935"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181299935" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181299935">(Nov 21 2019 at 12:37)</a>:</h4>
<p>The reason I asked the question is that my (old, but still possibly current) hypothesis was that ThinLTO was doing an inlining step here that is subsequently invalidated by a later incremental compile</p>



<a name="181300027"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181300027" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181300027">(Nov 21 2019 at 12:38)</a>:</h4>
<p>and that we fail to deduce that we cannot reuse the post-ThinLTO-optimized object because the import_map is failing to tell us about the dependence</p>



<a name="181300053"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181300053" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181300053">(Nov 21 2019 at 12:39)</a>:</h4>
<p>But I need to create my original work environment before I can provide better information</p>



<a name="181300985"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181300985" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181300985">(Nov 21 2019 at 12:50)</a>:</h4>
<blockquote>
<p>hypothesis was that ThinLTO was doing an inlining step here that is subsequently invalidated by a later incremental compile</p>
</blockquote>
<p>that's not what I found. it would be good to verify that you are really seeing something different than I.</p>



<a name="181631914"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181631914" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181631914">(Nov 22 2019 at 12:13)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span>  I want to spend a little bit of time on this, to try to get us in sync on our mutual understanding of the problem</p>



<a name="181631986"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181631986" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181631986">(Nov 22 2019 at 12:14)</a>:</h4>
<p>I can either: 1. try to recreate the steps I followed previously that led me to my earlier hypothesis, spelling out my thinking as I go in more detail. Or I could: 2. try to reproduce your result and in the process understand your thinking</p>



<a name="181632033"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181632033" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181632033">(Nov 22 2019 at 12:15)</a>:</h4>
<p>I do want to at least confirm: You <em>did</em> get to a point eventually where you were yourself  reproducing the whole problem locally, right?</p>



<a name="181632118"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181632118" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181632118">(Nov 22 2019 at 12:16)</a>:</h4>
<p>(it seems like you must have, based on the statements you were making. but I just want to be certain that you weren't drawing conclusions based on reading stuff I had posted.)</p>



<a name="181636631"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181636631" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181636631">(Nov 22 2019 at 13:22)</a>:</h4>
<p>Yes, I was able to reproduce the linker error mentioned in the GH issue:</p>
<div class="codehilite"><pre><span></span>= note: rust-lld: error: undefined symbol: _$LT$rubble..ble..time..Duration$u20$as$u20$core..fmt..Display$GT$::fmt::h343296729f83afc4
          &gt;&gt;&gt; referenced by time.rs:129 (src/ble/time.rs:129)
          &gt;&gt;&gt;               /home/jonas/dev/rubble/target/thumbv7em-none-eabi/debug/deps/rubble-a38354bf4561073f.3qtzuenakrd0qcaf.rcgu.o:(_$LT$$RF$T$u20$as$u20$core..fmt..Debug$GT$::fmt::h26ac05bd6f4f461e)
</pre></div>



<a name="181636665"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181636665" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181636665">(Nov 22 2019 at 13:23)</a>:</h4>
<p>I was able to do so with your reproduction repro, via <code>doit2.sh</code></p>



<a name="181636803"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181636803" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181636803">(Nov 22 2019 at 13:25)</a>:</h4>
<p>I added <code>-Zsymbol-mangling-version=v0 -Zhuman-readable-cgu-names</code> to all rustc invocations for better readability.</p>



<a name="181637502"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181637502" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181637502">(Nov 22 2019 at 13:35)</a>:</h4>
<p>okay. Oh, maybe the <code>-Zsymbol-mangling-version=v0</code> is why I've been getting different symbols than you</p>



<a name="181638143"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181638143" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181638143">(Nov 22 2019 at 13:43)</a>:</h4>
<p>definitely :)</p>



<a name="181638163"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181638163" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181638163">(Nov 22 2019 at 13:43)</a>:</h4>
<p>I don't know if we have an online demangler for the new scheme somewhere yet</p>



<a name="181638236"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181638236" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181638236">(Nov 22 2019 at 13:44)</a>:</h4>
<p>the <code>rustc-demangle</code> crate can handle it.</p>



<a name="181638322"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181638322" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181638322">(Nov 22 2019 at 13:45)</a>:</h4>
<p>it's better than the old scheme for debugging because symbol names actually contain information about generic arguments in a decodable form</p>



<a name="181641555"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181641555" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181641555">(Nov 22 2019 at 14:19)</a>:</h4>
<p>I've been sitting here wondering why my stage2 is taking so long to build. And then I remembered that in this build, I set llvm.optimize to false (to get assertions and try to improve my ability to use it with <code>rr</code>)</p>



<a name="181641620"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181641620" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181641620">(Nov 22 2019 at 14:20)</a>:</h4>
<p>I think stage1 rustc has been building librustc for 2.5 hours</p>



<a name="181642847"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181642847" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> davidtwco <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181642847">(Nov 22 2019 at 14:34)</a>:</h4>
<p>(having been interacting with rustc symbol mangling recently, I’ve been intending to try <a href="https://github.com/luser/rustfilt" target="_blank" title="https://github.com/luser/rustfilt">rustfilt</a> when I’m next looking at things, allegedly supports the new mangling scheme)</p>



<a name="181644981"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181644981" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181644981">(Nov 22 2019 at 14:59)</a>:</h4>
<p><span class="user-mention" data-user-id="116107">@davidtwco</span> <code>rustfilt</code> needs to update it's <code>rustc-demangle</code> version, but then it should work</p>



<a name="181807123"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181807123" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181807123">(Nov 25 2019 at 10:05)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> what is your schedule like today? Maybe we could pair-program on this a bit so that I can show you my observations?</p>



<a name="181807186"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181807186" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181807186">(Nov 25 2019 at 10:06)</a>:</h4>
<p>(the other options is that I could try to document them, as I described <a href="#narrow/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535/near/181631986" title="#narrow/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535/near/181631986">above</a>. But if we can talk live, that would probably be more efficient.)</p>



<a name="181808562"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181808562" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181808562">(Nov 25 2019 at 10:29)</a>:</h4>
<p>I'd could make time for this between 15:30 and 17:00 today.</p>



<a name="181809114"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181809114" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181809114">(Nov 25 2019 at 10:37)</a>:</h4>
<p>hmm, I have to pick up my son at school today, and I have to leave at like 15:45</p>



<a name="181809126"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181809126" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181809126">(Nov 25 2019 at 10:37)</a>:</h4>
<p>we could either try for a very short sync up today, or we wait for another day</p>



<a name="181809192"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181809192" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181809192">(Nov 25 2019 at 10:38)</a>:</h4>
<p>I will say that I still don't see the <em>exact</em> symbols that you are seeing in <a href="#narrow/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535/near/181113472" title="#narrow/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535/near/181113472">your code snippet</a>; but I see something similar enough (in terms of its code) that I do think we are talking about the same thing</p>



<a name="181809276"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181809276" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181809276">(Nov 25 2019 at 10:39)</a>:</h4>
<p>you asked there where that code was coming from. I had thought it was an artifact of the derived <code>fmt::Debug</code> impl for <code>Option&lt;Duration&gt;</code>, but I don't know if I actually verified that.</p>



<a name="181809762"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181809762" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181809762">(Nov 25 2019 at 10:45)</a>:</h4>
<blockquote>
<p>I see something similar enough (in terms of its code) that I do think we are talking about <em>the same thing</em></p>
</blockquote>
<p>(To be clear: I mean that we are talking about the same code snippet. But I don't think we've reached consensus on what is going wrong.)</p>



<a name="181810218"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181810218" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181810218">(Nov 25 2019 at 10:51)</a>:</h4>
<blockquote>
<p>you asked there where that code was coming from. I had thought it was an artifact of the derived <code>fmt::Debug</code> impl for <code>Option&lt;Duration&gt;</code>, but I don't know if I actually verified that.</p>
</blockquote>
<p>(Hmm, I guess if this were the case, then we would be finding this in one of the cgu's named <code>core.xxxx-in-rubble.yyyy-coremodname.volatile.*</code>. So that impression may be wrong.)</p>



<a name="181810472"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181810472" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181810472">(Nov 25 2019 at 10:55)</a>:</h4>
<p>but, oh: There is a call from <code>core.3u8ad80z-in-rubble.7rcbfp3g-option.volatile</code> over into the function we're discussing. From reading the .ll, it is indeed <del>called</del> referenced from the method <code>&lt;core::option::Option&lt;rubble::ble::Duration&gt; as core::fmt::Debug&gt;::fmt</code></p>



<a name="181818364"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181818364" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181818364">(Nov 25 2019 at 12:41)</a>:</h4>
<p>I can try to make it ~15:00 if that works for you</p>



<a name="181819202"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181819202" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181819202">(Nov 25 2019 at 12:52)</a>:</h4>
<p>Okay yeah if you can do 15:00 I think that would be good</p>



<a name="181910762"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181910762" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181910762">(Nov 26 2019 at 11:07)</a>:</h4>
<p>hey <span class="user-mention" data-user-id="124287">@mw</span> i have more questions</p>



<a name="181910786"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181910786" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181910786">(Nov 26 2019 at 11:07)</a>:</h4>
<p>in particular, after adding some more instrumentation and then looking at things more, I noticed this:</p>



<a name="181910848"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181910848" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181910848">(Nov 26 2019 at 11:08)</a>:</h4>
<p>during the first (clean) compile, we <em>do</em> observe from LLVM that the ble.volatile module imports the ble module: </p>
<div class="codehilite"><pre><span></span>[DEBUG rustc_codegen_llvm::back::lto] imported_module_callback importing_module_name: rubble.7rcbfp3g-ble.volatile imported_module_name: rubble.7rcbfp3g-ble
</pre></div>



<a name="181910913"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181910913" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181910913">(Nov 26 2019 at 11:09)</a>:</h4>
<p>but during the second compile (the one reusing incremental state, including the PostLTO optimized version of that cgu) , the data fed back from LLVM does not include that import information anymore (IOW the callback is not invoked for that importing/imported pair)</p>



<a name="181910942"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181910942" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181910942">(Nov 26 2019 at 11:10)</a>:</h4>
<p>which leads me to wonder: Who is in charge in this scenario of remembering that dependency link?</p>



<a name="181911009"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181911009" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181911009">(Nov 26 2019 at 11:10)</a>:</h4>
<p>since the whole point here is that the imported module <em>has</em> changed. But I think its the Rust's compilers job to know that, not LLVM's, right?</p>



<a name="181911186"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181911186" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181911186">(Nov 26 2019 at 11:13)</a>:</h4>
<p>Do we somehow seed the initial LLVM state with information about previous compilation sessions? Or should we be incorporating past dependencies into this map somehow?</p>



<a name="181914510"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181914510" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181914510">(Nov 26 2019 at 12:05)</a>:</h4>
<p>... Update: Using <code>rr</code> to step through the second LLVM run now. It looks like LLVM's <code>computeImportForFunction</code> may be failing to resolve the callee for the ble.volatile function's call.</p>



<a name="181915440"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181915440" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181915440">(Nov 26 2019 at 12:19)</a>:</h4>
<p>AH! Huge realization: I had one hand tied behind my back this whole time because every time I tried to pass along <code>-C llvm-args=...</code>, the bug went away</p>



<a name="181915469"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181915469" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181915469">(Nov 26 2019 at 12:20)</a>:</h4>
<p>But I just realized: Its because we (rightly) invalidate the previous incremental state if you don't pass the same set of llvm-args to <em>both</em> compiler invocations.</p>



<a name="181915515"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181915515" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181915515">(Nov 26 2019 at 12:20)</a>:</h4>
<p>So now I just pass the same extra settings to both invocations, and <em>boom</em>, I now can start getting debug output from LLVM!</p>



<a name="181919449"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181919449" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181919449">(Nov 26 2019 at 13:16)</a>:</h4>
<p>iirc, the ThinLTO import logic should always run on the full set of modules, so including the modules that we are going to re-use.</p>



<a name="181919533"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181919533" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181919533">(Nov 26 2019 at 13:17)</a>:</h4>
<p>So here's what LLVM debug info tells me so far:</p>



<a name="181919601"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181919601" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181919601">(Nov 26 2019 at 13:18)</a>:</h4>
<p>in the following, <code>rubble1</code> means the first (clean) compile, and <code>rubble2</code> means the second one that reuses previous work-products.</p>



<a name="181919605"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181919605" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181919605">(Nov 26 2019 at 13:18)</a>:</h4>
<p>Some <code>rubble1</code> output:</p>



<a name="181919627"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181919627" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181919627">(Nov 26 2019 at 13:18)</a>:</h4>
<div class="codehilite"><pre><span></span>* Module rubble.7rcbfp3g-ble.volatile exports 0 functions and 0 vars. Imports from 1 modules.
 - 1 functions imported from rubble.7rcbfp3g-ble
 - 0 global vars imported from rubble.7rcbfp3g-ble
</pre></div>



<a name="181919662"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181919662" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181919662">(Nov 26 2019 at 13:19)</a>:</h4>
<p>analogous <code>rubble2</code> output: </p>
<div class="codehilite"><pre><span></span>* Module rubble.7rcbfp3g-ble.volatile exports 0 functions and 0 vars. Imports from 0 modules.
</pre></div>



<a name="181919690"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181919690" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181919690">(Nov 26 2019 at 13:19)</a>:</h4>
<p>when I worked my way backwards to the place where these import lists were computed, I found this:</p>



<a name="181919732"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181919732" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181919732">(Nov 26 2019 at 13:19)</a>:</h4>
<p><code>rubble1</code> output:</p>
<div class="codehilite"><pre><span></span>Computing import for Module &#39;rubble.7rcbfp3g-ble.volatile&#39;
Initialize import for 544407980792537262 (_RNvXsL_NtCs2KcseFXz5HB_4core3fmtRNtNtCs4fqI2P2rA04_6rubble3ble8DurationNtB5_5Debug3fmtBz_)
 edge -&gt; 5161150766116740131 (_RNvXs_NtCs4fqI2P2rA04_6rubble3bleNtB4_8DurationNtNtCs2KcseFXz5HB_4core3fmt5Debug3fmt) Threshold:100
 edge -&gt; 10788096245104502577 (_RNvXNtCs4fqI2P2rA04_6rubble3bleNtB2_8DurationNtNtCs2KcseFXz5HB_4core3fmt7Display3fmt) Threshold:70
ignored! No qualifying callee with summary found.
</pre></div>



<a name="181919797"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181919797" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181919797">(Nov 26 2019 at 13:20)</a>:</h4>
<p>Analogous <code>rubble2</code> output:</p>
<div class="codehilite"><pre><span></span>Computing import for Module &#39;rubble.7rcbfp3g-ble.volatile&#39;
Initialize import for 544407980792537262 (_RNvXsL_NtCs2KcseFXz5HB_4core3fmtRNtNtCs4fqI2P2rA04_6rubble3ble8DurationNtB5_5Debug3fmtBz_)
 edge -&gt; 5161150766116740131 (_RNvXs_NtCs4fqI2P2rA04_6rubble3bleNtB4_8DurationNtNtCs2KcseFXz5HB_4core3fmt5Debug3fmt) Threshold:100
ignored! No qualifying callee with summary found.
</pre></div>



<a name="181919891"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181919891" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181919891">(Nov 26 2019 at 13:21)</a>:</h4>
<p>When I tried to step through the latter computation, it seemed like it was rejecting during <code>selectCallee</code> because of <code>ImportFailureReason::TooLarge</code></p>



<a name="181920055"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920055" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920055">(Nov 26 2019 at 13:23)</a>:</h4>
<p>But there's something super weird there. The call that its showing is the call as if the inlining didn't happen. I guess the import list construction is based on the LLVM IR prior to the optimization steps, even though we are going to end up deciding to reuse the PostLTO optimized cgu in the end?</p>



<a name="181920194"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920194" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920194">(Nov 26 2019 at 13:24)</a>:</h4>
<blockquote>
<p>iirc, the ThinLTO import logic should always run on the full set of modules, so including the modules that we are going to re-use.</p>
</blockquote>
<p>So basically, it looks to me like it <em>is</em> running on the modules we are re-using, but its making different decisions about what to construct for the Import List</p>



<a name="181920237"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920237" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920237">(Nov 26 2019 at 13:25)</a>:</h4>
<p>Which actually brings me to a question I've had in the back of my mind for the past hour or so...:</p>



<a name="181920504"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920504" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920504">(Nov 26 2019 at 13:28)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> What do you think of this theory as to the root bug here: <code>lto.rs</code>, when it decides which modules can be reused in PostLTO form, looks at the import-list and sees if they are all on the green modules list. But it is looking at the import list that it <em>currently</em> gets from LLVM. Should we be instead (or additionally) looking at the <em>old</em> import list, from <em>the time that we did the LTO optimization that we are reusing</em>? (Presumably we would just serialize the <code>ThinLTOImports</code> and re-load that, or add it to the dep-graph maybe.)</p>



<a name="181920520"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920520" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920520">(Nov 26 2019 at 13:28)</a>:</h4>
<p>hm, so the import list of <code>rubble1</code> above says that <code>rubble.ble.volatile</code> imports from <code>rubble.ble</code>, and <code>rubble.ble</code> clearly changes in <code>rubble2</code>, so my question is: why is <code>rubble.ble.volatile</code> marked as PostLTO reuse?</p>



<a name="181920551"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920551" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920551">(Nov 26 2019 at 13:29)</a>:</h4>
<p>we are only looking at the previous import list</p>



<a name="181920573"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920573" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920573">(Nov 26 2019 at 13:29)</a>:</h4>
<p>the new import list is just stored for subsequent sessions</p>



<a name="181920580"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920580" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920580">(Nov 26 2019 at 13:29)</a>:</h4>
<p>I don't think that's true?</p>



<a name="181920596"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920596" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920596">(Nov 26 2019 at 13:29)</a>:</h4>
<p>By "import list" I mean the <code>ThinLTOImports</code></p>



<a name="181920640"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920640" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920640">(Nov 26 2019 at 13:30)</a>:</h4>
<p>those are never serialized today AFAICT</p>



<a name="181920696"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920696" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920696">(Nov 26 2019 at 13:30)</a>:</h4>
<p>(let me just make sure this hasn't changed in the last month)</p>



<a name="181920736"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920736" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920736">(Nov 26 2019 at 13:31)</a>:</h4>
<p>maybe I'm misremembering. let me take a closer look</p>



<a name="181920763"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181920763" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181920763">(Nov 26 2019 at 13:31)</a>:</h4>
<p>so this <a href="https://github.com/rust-lang/rust/blob/0f6f66fcdc4abf110171ee06b1a72bdd2883b74f/src/librustc_codegen_llvm/back/lto.rs#L474" target="_blank" title="https://github.com/rust-lang/rust/blob/0f6f66fcdc4abf110171ee06b1a72bdd2883b74f/src/librustc_codegen_llvm/back/lto.rs#L474"><code>import_map</code></a> is relying on <a href="https://github.com/rust-lang/rust/blob/0f6f66fcdc4abf110171ee06b1a72bdd2883b74f/src/librustc_codegen_llvm/back/lto.rs#L854" target="_blank" title="https://github.com/rust-lang/rust/blob/0f6f66fcdc4abf110171ee06b1a72bdd2883b74f/src/librustc_codegen_llvm/back/lto.rs#L854">this LLVM call</a></p>



<a name="181921169"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181921169" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181921169">(Nov 26 2019 at 13:36)</a>:</h4>
<p>Yes, I think what I was talking about was how I did things in the first version of the PR implementing all of this. but  that was missing out on some re-use. Maybe we need a combination of both? I need to think more thoroughly about this...</p>



<a name="181922735"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181922735" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181922735">(Nov 26 2019 at 13:55)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> would it be plausible for the incr_comp_session_dir to store, for each LTO-optimized work-product <code>M</code>, the value associated in the <code>import_map</code> for key <code>M</code> (i.e. <code>import_map[M]</code>)?</p>



<a name="181922919"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181922919" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181922919">(Nov 26 2019 at 13:57)</a>:</h4>
<p>Yeah, I had already implemented that in <a href="https://github.com/rust-lang/rust/pull/53673" target="_blank" title="https://github.com/rust-lang/rust/pull/53673">https://github.com/rust-lang/rust/pull/53673</a>. Alex talked me out of it <code>;)</code></p>



<a name="181923016"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181923016" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181923016">(Nov 26 2019 at 13:58)</a>:</h4>
<p>does that mean <a href="https://github.com/rust-lang/rust/pull/53673#discussion_r212702032" target="_blank" title="https://github.com/rust-lang/rust/pull/53673#discussion_r212702032">this comment from alex</a> ?</p>



<a name="181923062"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181923062" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181923062">(Nov 26 2019 at 13:59)</a>:</h4>
<p>I'm willing to treat this bug as a concrete counter-example. Though of course I'd prefer a smaller example... :(</p>



<a name="181923102"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181923102" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181923102">(Nov 26 2019 at 13:59)</a>:</h4>
<p>Given all the information you've gathered so far, I have the feeling that I would understand exactly what's going if I sat down and thought about this for half an hour in peace and quiet. Unfortunately I need to run now <code>:(</code></p>



<a name="181923179"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181923179" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181923179">(Nov 26 2019 at 14:00)</a>:</h4>
<p>But if you want, you can assign the bug to me.</p>



<a name="181923305"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181923305" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181923305">(Nov 26 2019 at 14:01)</a>:</h4>
<blockquote>
<p>does that mean <a href="https://github.com/rust-lang/rust/pull/53673#discussion_r212702032" target="_blank" title="https://github.com/rust-lang/rust/pull/53673#discussion_r212702032">this comment from alex</a> ?</p>
</blockquote>
<p>rather this comment, I think: <a href="https://github.com/rust-lang/rust/pull/53673#discussion_r212708336" target="_blank" title="https://github.com/rust-lang/rust/pull/53673#discussion_r212708336">https://github.com/rust-lang/rust/pull/53673#discussion_r212708336</a></p>



<a name="181923564"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181923564" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181923564">(Nov 26 2019 at 14:03)</a>:</h4>
<p>But I still haven't paged in all the logic here -- having just discovered that some of my "knowledge" is in fact based on a version of the PR that never landed.</p>



<a name="181926772"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181926772" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181926772">(Nov 26 2019 at 14:38)</a>:</h4>
<blockquote>
<p>Yeah, I had already implemented that in <a href="https://github.com/rust-lang/rust/pull/53673" target="_blank" title="https://github.com/rust-lang/rust/pull/53673">https://github.com/rust-lang/rust/pull/53673</a>. Alex talked me out of it <code>;)</code></p>
</blockquote>
<p>I'm guessing the answer here is "No", but: Do you still have the original code for this lying around somewhere, for me to look at and maybe try out myself, rather than trying to reimplement it myself?</p>



<a name="181955069"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/181955069" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#181955069">(Nov 26 2019 at 19:27)</a>:</h4>
<p>the github comments are still referencing it, so it might be just a matter of a few clicks?</p>



<a name="182007550"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182007550" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182007550">(Nov 27 2019 at 10:51)</a>:</h4>
<p>The github comments say "outdated code" and when I tried to follow them, I got one of those pages that says "We went looking everywhere, but couldn’t find those commits."</p>



<a name="182055006"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182055006" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> bjorn3 <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182055006">(Nov 27 2019 at 20:16)</a>:</h4>
<p>The force-push messages have working before and after commit links in them.</p>



<a name="182096234"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182096234" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182096234">(Nov 28 2019 at 10:06)</a>:</h4>
<p>ah thanks <span class="user-mention" data-user-id="133247">@bjorn3</span> , I've always overlooked that</p>



<a name="182188404"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182188404" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182188404">(Nov 29 2019 at 16:05)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> okay I think I have a patch for this</p>



<a name="182188488"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182188488" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182188488">(Nov 29 2019 at 16:06)</a>:</h4>
<p>I'm going to do some internal testing, but here is the relevant <a href="https://github.com/pnkfelix/rust/commit/fe85919c74686a0de188dd4900467cf40bab73f5" target="_blank" title="https://github.com/pnkfelix/rust/commit/fe85919c74686a0de188dd4900467cf40bab73f5">commit</a>; it is largely inspired by the approach you referenced in that earlier PR.</p>



<a name="182322558"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182322558" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182322558">(Dec 02 2019 at 07:56)</a>:</h4>
<p>Great, I'll put looking at that on my todo list</p>



<a name="182331935"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182331935" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182331935">(Dec 02 2019 at 10:25)</a>:</h4>
<p>at this point I'm going to spend some time today exploring whether I can make a more robust regression test</p>



<a name="182336017"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182336017" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182336017">(Dec 02 2019 at 11:26)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> am I correct that the tests in <code>src/incremental/thinlto</code> are supposedly exercising ThinLTO because they have <code>-O</code> in their <code>// compile-flags</code> ?</p>



<a name="182339008"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182339008" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182339008">(Dec 02 2019 at 12:17)</a>:</h4>
<p>yes</p>



<a name="182344993"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182344993" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182344993">(Dec 02 2019 at 13:33)</a>:</h4>
<p>I'm not sure I'm going to be able to construct a (robust) test case for this. Even with  the knowledge of the bug that I've garnered, I really don't know how to reliably get LLVM to reproduce the at-LTO-time inlining behavior that we need to witness on my local machine.</p>



<a name="182345484"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182345484" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182345484">(Dec 02 2019 at 13:37)</a>:</h4>
<p>small functions like <code>fn foo() -&gt; u32 { 0 }</code> are imported very reliably by ThinLTO</p>



<a name="182345566"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182345566" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182345566">(Dec 02 2019 at 13:38)</a>:</h4>
<p>and functions with <code>#[inline(never)]</code> are also reliably not imported by ThinLTO, afaik</p>



<a name="182345582"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182345582" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182345582">(Dec 02 2019 at 13:38)</a>:</h4>
<p>maybe that helps?</p>



<a name="182346104"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182346104" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182346104">(Dec 02 2019 at 13:44)</a>:</h4>
<p>The scenario I need to recreate is one where we change our mind about whether to mark a fn as internal or not</p>



<a name="182346147"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182346147" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182346147">(Dec 02 2019 at 13:45)</a>:</h4>
<p>while at the same time, LTO has inlined the reference to that (to-be-marked internal) fn into some cgu that won't be rebuilt under the old logic</p>



<a name="182346159"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182346159" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182346159">(Dec 02 2019 at 13:45)</a>:</h4>
<p>I'm able to get the first step down pretty reliably</p>



<a name="182346185"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182346185" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182346185">(Dec 02 2019 at 13:45)</a>:</h4>
<p>but I haven't made much luck with the second part.</p>



<a name="182346280"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182346280" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182346280">(Dec 02 2019 at 13:46)</a>:</h4>
<p>I guess I could try inspecting the generated intermediate products from the LTO passes to try to figure things out more</p>



<a name="182360321"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182360321" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182360321">(Dec 02 2019 at 16:07)</a>:</h4>
<p>is there an <code>#[inline(..)]</code> attribute with the effect of "inline at link-time if possible, but not earlier." ?</p>



<a name="182362196"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182362196" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182362196">(Dec 02 2019 at 16:25)</a>:</h4>
<p>nope</p>



<a name="182362297"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182362297" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182362297">(Dec 02 2019 at 16:26)</a>:</h4>
<p>but functions without an <code>#[inline]</code> attribute are guaranteed to not be inlined across CGU boundaries before ThinLTO</p>



<a name="182362398"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182362398" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182362398">(Dec 02 2019 at 16:27)</a>:</h4>
<p>and you can rely on different <code>mod</code>s to end up as separate CGUs (when compiling incrementally), even if they are defined in the same source file.</p>



<a name="182363695"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182363695" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182363695">(Dec 02 2019 at 16:42)</a>:</h4>
<p>yeah, i've been making use of the latter property</p>



<a name="182363834"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182363834" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182363834">(Dec 02 2019 at 16:43)</a>:</h4>
<p>(the test infrastructure is impressive. The main thing missing is docs explaining how the tests work in the first place.)</p>



<a name="182555195"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182555195" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182555195">(Dec 04 2019 at 13:31)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> I decided to give up on finding a more robust test</p>



<a name="182555251"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182555251" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182555251">(Dec 04 2019 at 13:32)</a>:</h4>
<p>I filed PR <a href="https://github.com/rust-lang/rust/issues/67020" target="_blank" title="https://github.com/rust-lang/rust/issues/67020">#67020</a>. It still has a test, it just embeds the one we've already been using as a run-make test for the corresponding thumb target.</p>



<a name="182565781"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182565781" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182565781">(Dec 04 2019 at 15:25)</a>:</h4>
<p>I'll try to review tomorrow before the triage meeting</p>



<a name="182656310"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182656310" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182656310">(Dec 05 2019 at 12:57)</a>:</h4>
<p>thanks for your input. I really wish we could get a smaller test case</p>



<a name="182656388"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182656388" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182656388">(Dec 05 2019 at 12:58)</a>:</h4>
<p>I keep musing about trying to dive deeper into my understanding of LLVM optimization passes, with the thought that if I knew the right <code>-C llvm-opts=...</code> to feed in, I could replicate this more readily (and more generally)</p>



<a name="182656435"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182656435" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182656435">(Dec 05 2019 at 12:59)</a>:</h4>
<p>When you write "Yes, but the import map is always re-computed from scratch from the entire set of modules, including the older ones", do you mean that the import map somehow uses the state of the compiled <em>object code</em> ?</p>



<a name="182656630"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182656630" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182656630">(Dec 05 2019 at 13:01)</a>:</h4>
<p>another option here, which may be unpalatable, but <strong>would</strong> satisfy your desire to ensure we don't rely on anything beyond the previous serialized import map: for each CGU, if there is any change <strong>at all</strong> to the <del>import map</del> imports for the current (green) module, just do not reuse the PostLTO object file</p>



<a name="182656718"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182656718" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182656718">(Dec 05 2019 at 13:02)</a>:</h4>
<p>given that the import map seems to very rarely change for green modules, this condition seems like it would fire so rarely that it would not regress anything.</p>



<a name="182656763"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182656763" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182656763">(Dec 05 2019 at 13:03)</a>:</h4>
<p>oh, maybe this is what you meant by " If the two things match up" ?</p>



<a name="182656795"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182656795" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182656795">(Dec 05 2019 at 13:03)</a>:</h4>
<blockquote>
<p>When you write "Yes, but the import map is always re-computed from scratch from the entire set of modules, including the older ones", do you mean that the import map somehow uses the state of the compiled <em>object code</em> ?</p>
</blockquote>
<p>the import map is based on the state of the pre-optimization LLVM bitcode (which we also keep around for cached modules)</p>



<a name="182656956"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182656956" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182656956">(Dec 05 2019 at 13:05)</a>:</h4>
<blockquote>
<p>another option here, which may be unpalatable, but <strong>would</strong> satisfy your desire to ensure we don't rely on anything beyond the previous serialized import map: for each CGU, if there is any change <strong>at all</strong> to the import map, just do not reuse the PostLTO object file</p>
</blockquote>
<p>I need to think about that. It might be the correct fix actually</p>



<a name="182656969"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182656969" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182656969">(Dec 05 2019 at 13:06)</a>:</h4>
<p>I would have no problem adopting the latter.</p>



<a name="182657015"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657015" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657015">(Dec 05 2019 at 13:06)</a>:</h4>
<p>and I think I could hack it up fast</p>



<a name="182657044"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657044" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657044">(Dec 05 2019 at 13:06)</a>:</h4>
<p>just set-equality; so sort both slices and compare them for equality. easy peasy.</p>



<a name="182657071"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657071" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657071">(Dec 05 2019 at 13:07)</a>:</h4>
<p>I can believe that it is correct and clean because it sounds like a manual re-implementation of the red-green algorithm</p>



<a name="182657105"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657105" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657105">(Dec 05 2019 at 13:07)</a>:</h4>
<p>okay cool, i prefer this</p>



<a name="182657167"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657167" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657167">(Dec 05 2019 at 13:08)</a>:</h4>
<p>and then only save the "current" version of the import map?</p>



<a name="182657247"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657247" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657247">(Dec 05 2019 at 13:09)</a>:</h4>
<p>exactly</p>



<a name="182657297"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657297" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657297">(Dec 05 2019 at 13:10)</a>:</h4>
<p>that sounds good then</p>



<a name="182657308"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657308" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657308">(Dec 05 2019 at 13:10)</a>:</h4>
<p>let me go ahead and give it a whirl. I just finished pushing a new version of the PR with the test fixed in a couple ways</p>



<a name="182657336"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657336" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657336">(Dec 05 2019 at 13:10)</a>:</h4>
<p>oh but I guess I need to do triage for the rustc meeting first (!)</p>



<a name="182657345"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657345" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657345">(Dec 05 2019 at 13:11)</a>:</h4>
<p>i'll follow up on this after the meeting.</p>



<a name="182657356"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657356" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657356">(Dec 05 2019 at 13:11)</a>:</h4>
<p>(or when I finish triage; whichever comes first)</p>



<a name="182657360"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657360" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657360">(Dec 05 2019 at 13:11)</a>:</h4>
<p>this bug is a cautionary tale about trying to do incremental compilation outside of the well-tested framework <code>:)</code></p>



<a name="182657372"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657372" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657372">(Dec 05 2019 at 13:11)</a>:</h4>
<p>no rush <code>:)</code></p>



<a name="182657375"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657375" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657375">(Dec 05 2019 at 13:11)</a>:</h4>
<p>its definitely getting on my list of "favorite bugs" now, though</p>



<a name="182657387"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657387" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657387">(Dec 05 2019 at 13:11)</a>:</h4>
<p>right up there with some great GC bugs I tracked down back in the day</p>



<a name="182657468"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657468" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657468">(Dec 05 2019 at 13:12)</a>:</h4>
<p>It also has led me to think harder about how to go about testing incremental in the first place</p>



<a name="182657477"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657477" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657477">(Dec 05 2019 at 13:12)</a>:</h4>
<p>my favorite is still a random memory corruption/dangling pointer bug in LLVM that only rustc triggered :)</p>



<a name="182657520"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657520" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657520">(Dec 05 2019 at 13:13)</a>:</h4>
<p>this was my attempt to create a test that would expose the bug for me locally: <a href="https://gist.github.com/pnkfelix/8a5870135def2f6c46bbd37feed70e2c" target="_blank" title="https://gist.github.com/pnkfelix/8a5870135def2f6c46bbd37feed70e2c">https://gist.github.com/pnkfelix/8a5870135def2f6c46bbd37feed70e2c</a></p>



<a name="182657605"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657605" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657605">(Dec 05 2019 at 13:14)</a>:</h4>
<p>I was basically trying to manually enumerate as many ways to combine calls between modules that I could think of</p>



<a name="182657800"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182657800" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182657800">(Dec 05 2019 at 13:17)</a>:</h4>
<p>I'll probably give this a quick try tomorrow morning, just from the scenario description in your PR.</p>



<a name="182658069"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182658069" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182658069">(Dec 05 2019 at 13:20)</a>:</h4>
<blockquote>
<p>I'll probably give this a quick try tomorrow morning, just from the scenario description in your PR.</p>
</blockquote>
<p>making a test, you mean? That would be great.</p>



<a name="182707560"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182707560" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182707560">(Dec 05 2019 at 21:39)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> FYI I have updated the PR with the simpler strategy. It makes me happy.</p>



<a name="182749019"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749019" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749019">(Dec 06 2019 at 10:51)</a>:</h4>
<p>I'm so proud: <a href="https://github.com/rust-lang/rust/pull/67020#issuecomment-562526208" target="_blank" title="https://github.com/rust-lang/rust/pull/67020#issuecomment-562526208">https://github.com/rust-lang/rust/pull/67020#issuecomment-562526208</a></p>



<a name="182749096"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749096" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749096">(Dec 06 2019 at 10:52)</a>:</h4>
<p>Your description of the failure scenario is spot on</p>



<a name="182749333"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749333" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749333">(Dec 06 2019 at 10:55)</a>:</h4>
<p>I cannot say how happy I am you found that</p>



<a name="182749353"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749353" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749353">(Dec 06 2019 at 10:55)</a>:</h4>
<p>import-instr-limit seems important. :)</p>



<a name="182749366"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749366" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749366">(Dec 06 2019 at 10:55)</a>:</h4>
<p>(as in, I should learn about it. it may have made my life easier)</p>



<a name="182749422"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749422" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749422">(Dec 06 2019 at 10:56)</a>:</h4>
<p>it should make the test more stable, although I don;t think it is needed with the current defaults</p>



<a name="182749438"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749438" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749438">(Dec 06 2019 at 10:56)</a>:</h4>
<p>I only found out about it recently too</p>



<a name="182749471"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749471" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749471">(Dec 06 2019 at 10:57)</a>:</h4>
<p>the fact that the intermediate function needs to be a certain size was something I was not paying attention to in my attempts to replicate</p>



<a name="182749483"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749483" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749483">(Dec 06 2019 at 10:57)</a>:</h4>
<p>it needs to be not too big, but not too small either, right?</p>



<a name="182749490"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749490" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749490">(Dec 06 2019 at 10:57)</a>:</h4>
<p>(it can massively speed up optimized compilation in exchange for slightly lower runtime performance: <a href="https://github.com/rust-lang/rust/pull/66625" target="_blank" title="https://github.com/rust-lang/rust/pull/66625">https://github.com/rust-lang/rust/pull/66625</a>)</p>



<a name="182749494"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749494" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749494">(Dec 06 2019 at 10:57)</a>:</h4>
<p>(I'm speaking of <code>foo::bar</code>, here)</p>



<a name="182749588"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182749588" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182749588">(Dec 06 2019 at 10:58)</a>:</h4>
<p>Since it is an <code>internal</code> function in cfail2 and LLVM knows that there is only a single call site, it will probably inline it even if it gets very big</p>



<a name="182759314"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182759314" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182759314">(Dec 06 2019 at 13:21)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> I assume we'd all be happiest if I incorporate your new test into my PR. <span aria-label="smiley" class="emoji emoji-1f603" role="img" title="smiley">:smiley:</span></p>



<a name="182760974"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/182760974" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#182760974">(Dec 06 2019 at 13:41)</a>:</h4>
<p>yes please</p>



<a name="183115529"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/183115529" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#183115529">(Dec 10 2019 at 23:59)</a>:</h4>
<p>Very nice compilation time speedup! I would love to see that in CI.</p>



<a name="183206585"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/183206585" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#183206585">(Dec 11 2019 at 22:00)</a>:</h4>
<blockquote>
<p>Very nice compilation time speedup! I would love to see that in CI.</p>
</blockquote>
<p><span class="user-mention" data-user-id="239881">@Josh Triplett</span> this was aimed at a different topic, no?</p>



<a name="183206771"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/131828-t-compiler/topic/internalize_symbols%20and%20incremental%20artifacts%20%2359535/near/183206771" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/131828-t-compiler/topic/internalize_symbols.20and.20incremental.20artifacts.20.2359535.html#183206771">(Dec 11 2019 at 22:02)</a>:</h4>
<p>No, it was related to <a href="https://github.com/rust-lang/rust/pull/66625" target="_blank" title="https://github.com/rust-lang/rust/pull/66625">https://github.com/rust-lang/rust/pull/66625</a> posted above.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>